Many systems are controlled by machine learning systems. A common issue with such systems, however, is that when there is an anomalous outcome, such as a system failure, unexpected action, etc., there is no way to know why the system acted in the manner in which it did. For example, in a machine learning system to detect letters or numbers in images, tens or hundreds of thousands of training data (e.g., pictures along with coded letters or numbers) might be used to train the system. The system can then be used to act on incoming images to find letters and numbers in those images. At times, those outcomes might be anomalous. For example, the system may “find” letters or numbers that are not actually in the images, find incorrect letters or numbers, or fail to find letters or numbers in the images.
An issue with such systems, especially in the face of such anomalous results, is that it is difficult, if not impossible, to determine what training data caused the system to act anomalously. Therefore, it might not be possible to remove training data that caused the anomalous result from the machine learning model.
Techniques herein address these issues.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.